Introduction
Me
- Primary developer of
rtables >= 0.2.0 - Statistical Computing Consultant
- PhD in Statistical Computing
- Frequent collaborator with R core team
- several new features incorporated into R
R Tables For Regulatory Submission (RTRS) Working Group
- RConsortium Formal Working Group
- Members represent
- Multiple Pharma Companies
- Multiple table package authors
- US FDA
- RStudio
- Ongoing work to assess full feature-space Pharma Tables
- Collectively Authoring “State of the Field” style lit. review
RTRS-WG Call For Difficult Tables
File issues at:
https://github.com/RConsortium/rtrs-wg/issues
rtables
rtables - What
R package:
- Purpose built to create reporting tables
- General across table types
- Modern expressive API
rtables - Why
Cornerstone piece of larger work to enable clinical trial work in R
- Tables aren’t enough by themselves, but
- Can’t file without tables
rtables - Impact at Roche
196 Internal Production Table Variant Templates Across 24 Categories
Production Table Templates ———————————————————————————————————————————————————————— Total 196 Adverse Event 54 Concomittant Medication 13 Demographics and Baseline 5 ECG Related 7 Lab Test 38 Other 39 Response-Related 10 Statistical Model Summary 24 Time To Event 6
rtables Usage In Practice
- In use in multiple active Roche trials
- Will be used in any filings from these trials
- (Planned) All new Roche studies starting in 2023
- Powers exploratory work across many programs
- Including those not currently using it for regulator tables
Availability
rtables is
- Open Source with Permissive License
- Available on CRAN
- Developed in Public
- Funded and Copyright F. Hoffmann-La Roche AG
rtables introduction
rtables Is General But Informed By Pharma’s Needs
- Complex Table Structures
- Row & Column Space
- Row-group Summaries
- “Top-left” Annotations
- Titles and Footers
- Referential Footnotes
- Pagination
- Alternate Patient Counts
Consider an AE Table
Adverse Events
By Arm, Biomarker Load and Grade
———————————————————————————————————————————————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
Low High Low High Low High
(N=78) (N=68) (N=81) (N=73) (N=159) (N=141)
———————————————————————————————————————————————————————————————————————————————————————————————————————————
Patients with >0 events 62 (79.5%) 59 (86.8%) 78 (96.3%) 72 (98.6%) 140 (88.1%) 131 (92.9%)
Total events 1042 1018 571 569 1613 1587
--Any Grade-- 62 59 78 72 140 131
1 22 17 14 19 36 36
2 40 42 64 53 104 95
NERVOUS SYSTEM DISORDERS
--Any Grade-- 62 59 75 70 137 129
1 0 0 0 0 0 0
2 62 59 75 70 137 129
HEADACHE
--Any Grade-- 62 59 75 70 137 129
1 0 0 0 0 0 0
2 62 59 75 70 137 129
VASCULAR DISORDERS
--Any Grade-- 62 59 75 71 137 130
1 42 33 35 46 77 79
2 20 26 40 25 60 51
HYPOTENSION
--Any Grade-- 61 57 62 67 123 124
1 61 57 62 67 123 124
2 0 0 0 0 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 60 59 62 55 122 114
1 0 0 0 0 0 0
2 60 59 62 55 122 114
Starting Simple
l <-basic_table() %>%
analyze("USUBJID", afun = s_events_patients)
build_table(l, ADAE2, alt_counts_df = ADSL2) all obs
—————————————————————————————————————
Patients with >0 events 271 (90.3%)
Total events 3200
Splitting By Arm
l <-basic_table() %>%
split_cols_by("ARM") %>%
analyze("USUBJID", afun = s_events_patients)
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B
———————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
Counting By Grade
l <-basic_table() %>%
split_cols_by("ARM") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B
———————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
--Any Grade-- 121 150
1 39 33
2 82 117
Splitting By System Organ Class
l <-basic_table() %>%
split_cols_by("ARM") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B
————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145
1 0 0
2 121 145
VASCULAR DISORDERS
--Any Grade-- 121 146
1 75 81
2 46 65
Splitting By Preffered Term Within SOC
l <-basic_table() %>%
split_cols_by("ARM") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B
—————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
NERVOUS SYSTEM DISORDERS
HEADACHE
--Any Grade-- 121 145
1 0 0
2 121 145
VASCULAR DISORDERS
HYPOTENSION
--Any Grade-- 118 129
1 118 129
2 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117
1 0 0
2 119 117
Summarizing SOC
l <-basic_table() %>%
split_cols_by("ARM") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B
———————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145
1 0 0
2 121 145
HEADACHE
--Any Grade-- 121 145
1 0 0
2 121 145
VASCULAR DISORDERS
--Any Grade-- 121 146
1 75 81
2 46 65
HYPOTENSION
--Any Grade-- 118 129
1 118 129
2 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117
1 0 0
2 119 117
Adding an All Patients Column
l <-basic_table() %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)) %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2) ARM A ARM B All Patients
——————————————————————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%) 271 (90.3%)
Total events 2060 1140 3200
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
HEADACHE
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
VASCULAR DISORDERS
--Any Grade-- 121 146 267
1 75 81 156
2 46 65 111
HYPOTENSION
--Any Grade-- 118 129 247
1 118 129 247
2 0 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117 236
1 0 0 0
2 119 117 236
Title/Footer Annotations
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)) %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
build_table(l, ADAE2, alt_counts_df = ADSL2)WILLOWWIND - Adverse Events By Grade
——————————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
——————————————————————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%) 271 (90.3%)
Total events 2060 1140 3200
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
HEADACHE
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
VASCULAR DISORDERS
--Any Grade-- 121 146 267
1 75 81 156
2 46 65 111
HYPOTENSION
--Any Grade-- 118 129 247
1 118 129 247
2 0 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117 236
1 0 0 0
2 119 117 236
——————————————————————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123Referential Footnotes
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)) %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
child_labels = "visible",
indent_mod = -1,
split_fun = trim_levels_in_group("AEDECOD")) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl_narrow <- build_table(l, ADAE2, alt_counts_df = ADSL2)
fnotes_at_path(tbl_narrow,
c("AEBODSYS", "NERVOUS SYSTEM DISORDERS",
"AEDECOD", "HEADACHE")) <- "Non-migraine"
tbl_narrowWILLOWWIND - Adverse Events By Grade
——————————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
——————————————————————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%) 271 (90.3%)
Total events 2060 1140 3200
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
HEADACHE {1}
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
VASCULAR DISORDERS
--Any Grade-- 121 146 267
1 75 81 156
2 46 65 111
HYPOTENSION
--Any Grade-- 118 129 247
1 118 129 247
2 0 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117 236
1 0 0 0
2 119 117 236
——————————————————————————————————————————————————————————————————————
{1} - Non-migraine
——————————————————————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123The Culmination
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)) %>%
split_cols_by("BMRKR") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
split_fun = trim_levels_in_group("AEDECOD"),
child_labels = "visible",
indent_mod = -1) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl <- build_table(l, ADAE2, alt_counts_df = ADSL2)
fnotes_at_path(tbl,
c("AEBODSYS", "NERVOUS SYSTEM DISORDERS",
"AEDECOD", "HEADACHE")) <- "Non-migraine"
tbl
The Table
WILLOWWIND - Adverse Events By Grade
———————————————————————————————————————————————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
Low High Low High Low High
———————————————————————————————————————————————————————————————————————————————————————————————————————————
Patients with >0 events 62 (79.5%) 59 (86.8%) 78 (96.3%) 72 (98.6%) 140 (88.1%) 131 (92.9%)
Total events 1042 1018 571 569 1613 1587
NERVOUS SYSTEM DISORDERS
--Any Grade-- 62 59 75 70 137 129
1 0 0 0 0 0 0
2 62 59 75 70 137 129
HEADACHE {1}
--Any Grade-- 62 59 75 70 137 129
1 0 0 0 0 0 0
2 62 59 75 70 137 129
VASCULAR DISORDERS
--Any Grade-- 62 59 75 71 137 130
1 42 33 35 46 77 79
2 20 26 40 25 60 51
HYPOTENSION
--Any Grade-- 61 57 62 67 123 124
1 61 57 62 67 123 124
2 0 0 0 0 0 0
ORTHOSTATIC HYPOTENSION
--Any Grade-- 60 59 62 55 122 114
1 0 0 0 0 0 0
2 60 59 62 55 122 114
———————————————————————————————————————————————————————————————————————————————————————————————————————————
{1} - Non-migraine
———————————————————————————————————————————————————————————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123
Table Layout Code Is Naturally Parameterized
Table Structure
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)
) %>%
split_cols_by("BMRKR") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
split_fun = trim_levels_in_group("AEDECOD"),
child_labels = "visible",
indent_mod = -1) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl <- build_table(l, ADAE2, alt_counts_df = ADSL2) Business Logic
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)
) %>%
split_cols_by("BMRKR") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
split_rows_by("AEBODSYS",
split_fun = trim_levels_in_group("AEDECOD"),
child_labels = "visible",
indent_mod = -1) %>%
summarize_row_groups("AETOXGR",
cfun = ids_per_grade) %>%
split_rows_by("AEDECOD") %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl <- build_table(l, ADAE2, alt_counts_df = ADSL2)Multi-target Tables
l <-basic_table() %>%
split_cols_by("ARM") %>%
summarize_row_groups("USUBJID",
cfun = s_events_patients) %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl <- build_table(l, ADAE2, alt_counts_df = ADSL2)
tbl
ARM A ARM B
———————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%)
Total events 2060 1140
--Any Grade-- 121 150
1 39 33
2 82 117
## for now helper I wrote, coming to rtables API soon
set_format_at_path(tbl, c("root",
"@content", "pat_count"),
format = "xx (xx.xx%)")
ARM A ARM B
—————————————————————————————————————————————————————
Patients with >0 events 121 (82.88%) 150 (97.40%)
Total events 2060 1140
--Any Grade-- 121 150
1 39 33
2 82 117
set_format_at_path(tbl, c("root",
"@content", "pat_count"),
format = "xx (xx.%)")
ARM A ARM B
———————————————————————————————————————————————
Patients with >0 events 121 (83%) 150 (97%)
Total events 2060 1140
--Any Grade-- 121 150
1 39 33
2 82 117
Pagination
pagtbl <- paginate_table(tbl_narrow, lpp = 35) pagtbl[[1]]
WILLOWWIND - Adverse Events By Grade
———————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
———————————————————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%) 271 (90.3%)
Total events 2060 1140 3200
NERVOUS SYSTEM DISORDERS
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
HEADACHE {1}
--Any Grade-- 121 145 266
1 0 0 0
2 121 145 266
VASCULAR DISORDERS
--Any Grade-- 121 146 267
1 75 81 156
2 46 65 111
HYPOTENSION
--Any Grade-- 118 129 247
1 118 129 247
2 0 0 0
———————————————————————————————————————————————————————————————————
{1} - Non-migraine
———————————————————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123pagtbl[[2]]
WILLOWWIND - Adverse Events By Grade
——————————————————————————————————————————————————————————————————————
ARM A ARM B All Patients
——————————————————————————————————————————————————————————————————————
Patients with >0 events 121 (82.9%) 150 (97.4%) 271 (90.3%)
Total events 2060 1140 3200
VASCULAR DISORDERS
--Any Grade-- 121 146 267
1 75 81 156
2 46 65 111
ORTHOSTATIC HYPOTENSION
--Any Grade-- 119 117 236
1 0 0 0
2 119 117 236
——————————————————————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123Page-by Splitting
l <-basic_table(title = "WILLOWWIND - Adverse Events By Grade",
prov_footer = wwind_prov_stamp()) %>%
split_cols_by("ARM",
split_fun = add_overall_level("All Patients",
first = FALSE)) %>%
split_rows_by("BMRKR", page_by=TRUE) %>%
split_rows_by("AEBODSYS",
split_fun = trim_levels_in_group("AEDECOD"),
child_labels = "visible",
indent_mod = -1) %>%
analyze("AETOXGR",
afun = ids_per_grade,
show_labels = "hidden")
tbl <- build_table(l, ADAE2, alt_counts_df = ADSL2)
ptbl <- paginate_table(tbl, lpp = 35)
ptbl[[1]]
WILLOWWIND - Adverse Events By Grade
BMRKR: Low
———————————————————————————————————————————————————————
ARM A ARM B All Patients
———————————————————————————————————————————————————————
NERVOUS SYSTEM DISORDERS
--Any Grade-- 62 75 137
1 0 0 0
2 62 75 137
VASCULAR DISORDERS
--Any Grade-- 62 75 137
1 42 35 77
2 20 40 60
———————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123ptbl[[2]]
WILLOWWIND - Adverse Events By Grade
BMRKR: High
———————————————————————————————————————————————————————
ARM A ARM B All Patients
———————————————————————————————————————————————————————
NERVOUS SYSTEM DISORDERS
--Any Grade-- 59 70 129
1 0 0 0
2 59 70 129
VASCULAR DISORDERS
--Any Grade-- 59 71 130
1 33 46 79
2 26 25 51
———————————————————————————————————————————————————————
file: /path/to/WILLOWWIND/aet05.R *** data snapshot: 2022-04-28 *** user: gb123Abstraction and Types of Effort
Types of Effort In Creating Tables
- Front Line Work - SPAs
- SPA-Enabling Develoment (SMEs)
- Core/general Tooling Development
SPAs
- Responsible for ultimate creation of tables
- Use templates and other tools provided to them
- Creation of Ad-hoc Tables
- “Last-Mile Delivery”
SMEs - SPA Enabling Tool Developers
- Develop table templates/functions
- Business and Statistical Logic
- for standard tables
Core Table Framework
- Provide building blocks and tools that SMEs use
- Not targeted specifically at any particular table endpoint
Largely Unsupported SPAs
Robust Spa-Enabling Dev Efforts
Basic Research/Innovation Supporting SMEs
Beware
Largely Unsupported SPAs
Robust Spa-Enabling Dev Efforts
Basic Research/Innovation Supporting SMEs
Research, Innovation, and Stakeholder Needs
The Apocryphal Henry Ford
rtables Is Not Faster Horses
- New (to R) way to make tables
- Result of novel statistical computing research
Three Pillars of How We Got Here
Management Support
Upper Management
- Support POs and Tech Leads
- Trust them to identify and pursue innovation
NEST Leadership
- Then Adrian Waddell (TL) and Tad Lewandowski (PO)
- Saw importance of tables as both need and opportunity
- Devoted NEST efforts to innovate in table space
- Narrow, applied research program
- within larger NEST product
- Now continued by Pawel Rucki (TL) and Jaime Pires (PO)
Stakeholders - SME Team
- Responsible for table template creation
- Formal goal of 200 tables during 2020
- Communicated what they needed, flexible on the how
- Willing to invest in learning new way of making tables
- resulting in invaluable feedback on API, capabilities, etc
Research - rtables Team
- Not responsible for delivery of any given table
- frees us to think about tables as a whole
- Asked SMEs what they need to be able to do
- not how it should let them do it
- Direct frequent collaboration with SME team
- tight feedback loop
- what works, what doesn’t, what’s still missing
Knock Down Any of the Pillars
rtables doesn’t end up where it is now.
Next Steps
Whats Next For rtables
- Collaboration with RStudio on tgen
- Multi output-format table renderer
- rtf, HTML, etc
- Visual formatting of tables
- color, bolding, etc
- Multi output-format table renderer
- New features of
rtables- QC-targeting table comparison functionality